Add Japanese v1.1 as a fourth build target - #777

The group JSONs under assets/ describe one version only, so a second
version needs a whole duplicate file selected by an ifeq in every .mk.
Nearly all of an asset is version independent though: of the Japanese
assets measured against their US counterparts, 97% differ in rom_offset
and nothing else.

Let an asset instead carry an override block keyed by version name,
holding only the keys that differ. new_extract_assets.py folds it in
before anything reads the asset, so the rest of the pipeline is
unchanged, and the version comes from one --version flag on
ASSET_EXTRACT rather than a switch in each .mk.

No asset carries an override yet, so this is a no-op.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Assets extract to one shared path regardless of version, and an asset
that already exists on disk is never extracted again. Building a second
version therefore reuses the first version's bytes, silently, and they
end up in the ROM. `make clean` does not help: it clears build/, not the
asset tree.

Record the version alongside the asset list in .assets-local.txt and
re-extract when it changes. The revision ID goes to 2 so an existing
tree is refreshed once rather than trusted.

This matters little for EU, where 31 assets carry their own offsets, and
a great deal for a version that differs more widely.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Torch generated into assets/code/<name> for every version, so building
one version left its generated C in the tree for the next version to
compile. Route it through ASSET_CODE_DIR, which the linker script now
receives as a define, and drop the assets/code/ prefix from the
rainbow_road includes so they resolve through the include path instead
of a fixed location. EU has no config.yml entry and is built on top of
the US assets, so it keeps reading the US output.

torch.hash.yml is not version aware and would skip regeneration after a
switch, so the assets target clears it first; Torch takes under a second.

Also mark extracted assets .PRECIOUS. They come out of the baserom, and
make's habit of deleting a target whose recipe failed leaves a tree that
only re-extraction can repair.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

macOS still ships GNU make 3.81 as `make`, and tools/Makefile generates
its per-tool rules with `define COMPILE =`, which is 3.82 syntax. Under
3.81 those rules do not take effect and make falls back to its built-in
%: %.c rule, so each tool is linked from its first source file alone.
n64graphics is built from n64graphics.c and utils.c, so it fails with an
undefined _g_verbosity rather than anything that points at the cause.

The docs already tell macOS users to build with gmake, but this line
then calls bare make and undoes that. This makefile needs 4.0+ for != in
the first place, so $(MAKE) is always a new enough make.

Makefile option and expected checksum only; the asset map and the code
differences follow.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

An asset is described once and usually only moves between versions, but a
few also change shape: seven menu textures are a different width in the
Japanese release, and one audio sequence a different length. Neither can
be said today, because meta is shared by every version.

Let meta hold an "overrides" map listing just the keys that differ for a
version. Nesting them keeps a version name from colliding with a real
meta key, and an asset with no override behaves exactly as before.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

877 assets carry a jp.v11 offset, and seven also carry a dims override
where the Japanese menu text is a different width than the US artwork.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

12379 group assets carry a jp.v11 override. 12060 of them differ from
the US entry in nothing but rom_offset, which is why these are overrides
rather than a second copy of every group file.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Registers the Japanese 1.1 cartridge by checksum with its own segment
table, and adds the yamls describing the segments it relocates.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The 39 tkmk00 menu textures the Japanese build needs are sliced straight
out of the cartridge, the same way the US blobs already are, so nothing
has to be checked in for them.

Offsets were derived by locating each blob in the cartridge; every one
occurs exactly once, and all 39 agree with the offsets recorded when the
match was originally worked out.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Every difference is behind #ifdef VERSION_JP, so the other three
versions compile exactly as before. Track path data moves out of
course_data.c into .inc.c files so the two versions can share it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

A tkmk00 block is described twice in assets.json at the same ROM offset:
the .tkmk00 blob under bin/ is the build input, and the .png beside it
under textures/ is a decoded copy. Extraction groups entries by offset
and then decided whether to decode by looking at the first one in the
group, so whichever was listed first won.

The .tkmk00 entries come first in the file, so all 63 of the textures/
PNGs were written straight from the compressed bytes and came out as
noise. They only ever looked right on an incremental run that already
had the blob on disk.

Decode alongside instead: the blob keeps the compressed bytes it needs
and the png gets the decoded image. Nothing in the build consumes those
PNGs, so no ROM changes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

JP ships a different build of this course. Its packed display-list
stream is disassembled back into Gfx macros here, so the JP lists are
ordinary committed source like every other course instead of a
checked-in blob.

The disassembly is verified two ways. Re-emitting US from the US blob
reproduces the committed course_displaylists.inc.c character for
character, and the JP output compiles and repacks to the 13726 bytes
that sit at rom 0x90604C in the cart.

A single #ifdef VERSION_JP after the includes selects the JP file, so
us, eu.v10 and eu.v11 compile from exactly the bytes they did before.
This replaces course_displaylists.jp.v11.s, which was never wired into
the build (ASM_DIRS does not cover courses/).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Group assets extract to one shared path regardless of version, guarded by
a `.export` sentinel that did not name the version. After building one
version the sentinel looks current, so building a second skips extraction
entirely and compiles the first version's artwork into the ROM.

That is silent: wrong artwork compiles fine, and the damage only shows up
as a ROM hash mismatch far downstream, cascaded by the size change.

It matters now because 194 assets genuinely differ in bytes between us and
jp.v11 rather than merely moving, which is a case EU never exercised: only
31 assets in the repo carry EU entries and EU art is US-identical.

Name the sentinel .export.$(VERSION), the same way .assets-local.txt
already records the extracted version for assets.json.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

An uncompressed asset uses the baserom itself as its source block, and
asset_from_block closes the block when it is done. That closes the handle
the caller still owns, so every later export_bin dies on a closed file.

extract_asset already worked around it by reopening, which is why images
survive and only raw `bin` assets fail. jp.v11 has enough of them to stop
the build outright.

Close the block only when it is a temporary one.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

course_offsets.c may guard its texture table with #ifdef VERSION_JP, and
the texture order in that table decides the segment 5 layout. The
generator already resolves the conditional, but nothing passed it the
version, so it always took the #else and a JP build linked its display
lists against the US texture addresses.

The generated files also live at one shared path while their contents are
version-specific, and course_offsets.c does not change when the built
version does. Stamp the version so a switch regenerates them.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Group assets extract to one shared path whatever the version is, guarded
by a .export sentinel that only depended on the group's json. Switching
version therefore kept the previous version's images and compiled them
into the ROM, silently: wrong artwork builds fine and only shows up as a
ROM hash mismatch, cascaded by the size change.

EU never exposed this. Its art is US-identical and merely relocated, so
US <-> EU has no asset whose bytes differ. jp.v11 has 194 that do.

Three things were needed, and each is useless alone:

  - Depend the sentinel on a stamp holding the version currently on disk.
    Naming the sentinel per version instead looks right but is not: it
    records that a version was extracted once, which stays true after the
    other version overwrites the shared files.
  - Give the image rules a real recipe rather than `;`. An empty recipe
    marks the target updated without re-stat'ing it, so the .bin and .mio0
    built from a re-extracted png kept the old bytes until a second make.
  - Extract EU's group assets from the US cart. None of the 12,521 group
    assets carries an eu offset, so reading the EU cart at US offsets
    returns garbage. EU relied on US files happening to be left on disk.
    ASSET_VERSION mirrors TORCH_VERSION, which folds EU to US already.

Verified by building jp.v11 -> us -> eu.v10 -> eu.v11 -> jp.v11 with no
cleaning: all three byte-match, and JP re-extracts its own assets after.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

JP ships its own audio: every bank differs, the sample table is 0x180
larger, and 5 of the 30 sequences are different music. Map them instead
of carrying the blobs.

The banks and sample table get their own entries, the way eu.v10 already
has bin/audiobanks.eu.bin, because their contents differ rather than
merely moving.

The sequences do not need that. 29 of the 30 already carried a jp.v11
offset, and extraction writes the built version's bytes to the shared
music/*.m64 path, which is what the other 25 sequences have always
relied on. So the five .ifdef VERSION_JP blocks pointing at .jp.v11.m64
paths were doing nothing the offset did not already do; drop them and
let all thirty read alike. 00_seq_00 needed the offset plus a size
override, being 0x50 bytes longer in JP.

Same for the onomatopoeia TLUT, which turns out to be identical in both
versions anyway: 512 values, byte for byte.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

JP shows three forms of each course name where US shows one: romanised
ascii, japanese, and japanese with one name shortened to fit. menu_items.c
already expected three files; nothing produced them.

Torch's mk64:metadata factory writes exactly one names table per config
entry, from the fixed `name` field, alongside nineteen other tables built
from the same course metadata, and its parser requires every field of
every course. Three names tables would therefore mean three copies of all
twenty-one fully populated course yamls, differing in a single string and
re-emitting nineteen identical tables each time. That is the per-version
duplication this branch removes everywhere else.

So the twenty names live in tools/jp_course_names.json and a small script
writes the three tables in EUC-JP. They are hand-authored map data exactly
like `name: banshee boardwalk` already is in the course yamls; they just do
not need the twenty-table machinery. If the factory ever takes the names
field from its config entry, the json folds back into yamls/courses and no
source changes.

Also defines the two symbols the JP link was missing. D_0B002900 is the
segment-0B texture US has at 0x0B002A00 and belongs beside it.
gCreditsTextExtra is different in kind and is called out in the file: JP's
mirror-mode credits table is not decompiled, so it is an address rather
than source, and it is the one place this target is not built from the
repo.

With these, jp.v11 links and produces a ROM. It does not match yet.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

JP packs both credit sets into one table: the english names occupy the
first 63 entries of gCreditsText and the japanese ones the second 63,
starting at エグゼクティブ プロデューサー. Mirror mode shows the japanese
half, which is what gCreditsTextExtra points at - the same table entered
at its midpoint rather than a second table.

It was a bare address, which read as data the repo does not build. It is
not: gCreditsText is ordinary source in src/ending/credits.c and this is
an offset into it. Say so, and derive the symbol from it.

The ROM is byte for byte identical either way.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Completes the jp.v11 side of assets.json. Every asset that needs a JP
offset now has one; what is left without is US-only or EU-only, and has
its own jp.v11 entry elsewhere.

Two groups. The 36 common textures share one MIO0 block, which sits at
0x1323A0 in JP against 0x132B50 in US and decompresses to the same
184664 bytes, so the layout is unchanged and every asset in it moves by a
constant 0xA40. Each was located by finding its bytes uniquely inside the
decompressed block and the delta checked across all 36.

The other 39 are the decoded copies of the tkmk00 blobs, sharing their
offsets, so they take the offsets the blobs already use.

Neither group is a build input: the ROM takes the common textures from
Torch's assets/code/<version>/common_data and the tkmk00 art from the
bin/ blobs. These entries are what make `extract_assets.py jp.v11` give a
JP player their own artwork instead of the American, which is the point
of the map.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The eight blue shell frames sit 0xAEE0 later in the JP cart, the same
shift the lakitu frames use. Each was matched by decompressing every MIO0
block in the region and comparing against the reference's extracted data,
so the shift is measured rather than carried over.

Their shared palette lives in the common textures block, which moves to
0x1323A0, but at the SAME offset within it. The block is not uniformly
shifted: the assets earlier in it do not move and the later ones drop by
0xA40, so a block_offset has to be checked per asset rather than adjusted
wholesale.

That matters here because these are ci8: the .bin the ROM gets is the png
round-tripped through its palette, so a wrong tlut corrupts the indices
even when the image offsets are right.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

A ci4/ci8 image reaches the ROM as png -> indices, remapped through its
palette. That is lossless only while every index the ROM uses is the first
one holding its colour. Several JP palettes repeat a colour, so the trip
back picks the earlier index and quietly rewrites the image: correct
offset, correct palette, wrong bytes. It cost 36 files across three
courses and 25,664 bytes of the ROM.

`type: bin` is the existing escape and is how the lakitu frames are done,
but it reads the ROM directly, so it only works where the asset is stored
uncompressed. These are MIO0 and it would copy compressed bytes. The
jp.v11 entries already said `"compressed": true` in the hope that
something honoured it; nothing did. Now export_bin decompresses the block
and reads from there.

That exposed the other half of the png rule bug: an asset extracted raw
has no png, so the .bin rule handed n64graphics a missing file, and make
deleted the .bin extraction had just written. The rule now depends on the
extraction and converts only when a png is really there.

Verified: those three courses go from 36 differing files to 0 against the
byte-perfect JP reference, and us/eu.v10/eu.v11 still byte-match.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

`make VERSION=jp.v11` cannot build after a us build: it dies converting
gTextureCactus1Left.png. Those five are extracted raw for jp.v11 because
their palette repeats a colour, and 370109612 guarded their rule on the
png being present. A us build leaves its png behind, so the guard passes,
n64graphics runs it against the JP palette, and it fails over the bytes
extraction had just written.

Extraction now removes a png it supersedes, so absence is the signal that
an asset is raw. The eight lakitu rules then have to stop naming the png
as a prerequisite, and take the shape the three course rules already
have. They were not failing before only because extraction runs at parse
time and leaves the .bin newer than the stale png: true today, but not
something a rule should rest on.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

yamls/jp.v11/common_data.yml was the us yaml with a different segment
address, so all 385 of its offsets still described the us layout. The two
versions do not share one: us puts the CPU behaviour tables and the
target speed tables ahead of the textures, JP puts the textures first and
packs both at the end of the segment. Every texture therefore sits 0xA40
lower than its us offset, and those 21 entries sit 0x23800 higher.

With the offsets corrected the built segment is byte-identical to the
cartridge's, 184,664 bytes, and the 98 textures that were wrong come out
of Torch already right. config.yml sets `sort: OFFSET`, so the entries do
not need reordering and this file stays comparable with the us one line
for line.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Segment 0x0B is the same 36,232 bytes in both versions and identical up
to 0x6BB8, but JP arranges the tail differently: the four
podium_ceremony_path tables move from the end of the segment to just
ahead of light1, and everything between them shifts up by 0x2E0.

yamls/jp.v11/ceremony_data.yml still described the us arrangement, so
Torch read the display lists out of what is texture data in JP. A `gfx`
entry is sized by scanning for its end opcode rather than from the yaml,
so each one ran past its real end and the segment came out 1,680 bytes
over. With the offsets corrected the built segment is byte-identical to
the cartridge's.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

JP ships different geometry for this course: 5,949 vertices against the
us table's 5,936, with 674 matching at the head and 1,254 at the tail.
That is the same difference the packed display lists carry, and it is
why they hold 434 differing gsSPVertex operands.

Separate file under one guard, the shape 6b464168e used for the display
lists. A file that currently produces byte-perfect us/eu.v10/eu.v11
cannot regress while the us table sits untouched on the #else branch.

The table was read from the course geography block in the JP cart with a
generator checked both ways against upstream's own us file: the us
source compiles to the same bytes the us build produces, and those bytes
re-emit that file character for character.

This is the last piece: make VERSION=jp.v11 now reproduces the cart.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Every version's ROM rule runs doxygen_symbol_gen.py, but the script
hardcodes build/us/mk64.us.map. A fresh clone that builds anything other
than us first therefore fails on the last line of the recipe, after the
ROM it just produced is already correct. This is reachable on master
today with `make VERSION=eu.v10`; it is not specific to the version this
branch adds.

The script now takes the map as an argument and still defaults to the us
one, so anything calling it bare keeps working.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

EU has no entry in config.yml and builds on the us Torch output, which is
what ASSET_CODE_DIR and ASSET_VERSION already arrange. The assets target
still passed $(BASEROM), so `make assets VERSION=eu.v11` handed Torch the
EU cart, matched no sha1, and silently wrote nothing; the link then failed
on a missing assets/code/us object.

Found by building the four versions in the other order from a fresh clone.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

134 assets carried a jp.v11 `type: bin` override on the theory that their
palette repeats a colour, so the png round trip would remap indices and
rewrite the image. Removing all of them and rebuilding says otherwise:
129 come out byte-identical through the normal image path. The override
was standing in for offsets that are now correct, not for a palette
problem.

The five kalimari cacti are real, and for a different reason. gTLUTCactus
has no jp.v11 entry and its palette does not appear anywhere in the JP
cart, so there is no palette to convert through. Those keep the retype,
and kalimari_desert.mk keeps the guard that goes with it. The other ten
.mk files return to upstream's plain rule.

All four ROMs still byte-identical, and the build log confirms the png
path really ran rather than reusing a leftover .bin.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

567 places had a guard block closing and the identical guard reopening on
the very next line, one display list entry at a time. Merging them removes
1,134 lines without touching a line of guarded code: the same entries
survive under the same condition, in the same order.

Only blocks with no `#else` are merged. With one, the merge would have to
interleave the two branches, and declaration order is data layout here.
The 33 pairs of that shape are left alone.

All four ROMs still byte-identical.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The note asked for the same arrangement in data, rodata and bss. Those
blocks are already there, so the TODO reads as unfinished work when
nothing is outstanding. Say what the block is for instead.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

README gains the fourth row. compiling.md gains a "Building JP" section
next to "Building EU", and it has to say two things that section does
not: Torch writes its generated C per version, so `make assets` needs the
version too, and unlike EU, JP does not need us built first. It extracts
everything from the Japanese cart, so a fresh clone builds it on its own.

first-diff.py and diff_settings.py learn `--jp11` / `-jp11`, so the
commands the new section points at exist.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

clang-format wraps this array differently now that the table is a line
longer. Whitespace only: the non-whitespace bytes are identical.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The wildcard was for the `.export.$(VERSION)` scheme this branch replaced
with one shared sentinel guarded by assets/.version. Nothing writes a
suffixed one now, so match the file that actually exists.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Revision 1.1 of the December 1996 Japanese release, not a 1997 version.
The neighbouring lines are 1997 releases and this one inherited that by
proximity.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

git-clang-format against 230fe2e with the pinned clang-format 14, so
only lines this branch added are considered. A stray blank line and one
continuation alignment.

The two other files it flags are left alone deliberately.
src/audio/external.c would be re-indented for the JP branch, which nests
a level deeper around a body both branches share, so the result is
correct for JP and wrong for us. src/menu_items.c holds EUC-JP text and
clang-format counts bytes rather than columns, so its wrapping there is
not meaningful.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Review feedback: the case labels do not need duplicating, only the two
assignments differ between versions. Same preprocessed output either way.

All four ROMs still byte-identical.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Both said the US revision replaced the JP code. It is the other way round: the
December 1996 JP 1.0 cartridge has yoshi_valley_cpu_path and the update_path_index
recovery chains, byte for byte the same as US, so JP 1.1 is the revision that
replaced them.

Comments only. jp.v11 still byte-identical.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>